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(54) System for delivering video data 

(57) A system for delivering video data comprises a 
video server (1) including a video database (11) for stor- 
ing video index information (13) defining a plurality of 
settings for a transfer bit rate of video data and indicat- 
ing which data included in original video data (12) the 
video server should transfer to a client (2) through a net- 
work (3) when setting the transfer bit rate to one of the 
plurality of settings, a video data assembler (14) for 
extracting data from the original video data by referring 
to the video data index information so as to set the 
transfer bit rate to one of the plurality of settings, and for 
reassembling the extracted data to create video data to 
be transferred at the set transfer bit rate to the client, 
while maintaining consistency in contents of the original 
video data, and a video data delivery unit (15) for deliv- 
ering a constant amount of data at certain intervals in 
accordance with the set transfer bit rate in order to 
transfer the video data reassembled by the video data 
assembler to the client. Furthermore, the video server 
comprises a network load sensor (17) for measuring a 
load imposed on the network. 
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Description 

BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to a system for deliv- 
ering video data through a network. 

Description of the Prior Art 

Improvements in hardware and software tech- 
niques have provided a way to handle data such as 
video data, which are difficult to handle as digital data, 
in the same manner as in the case of numerical data. 
Furthermore, as Internet and Intranet have become 
widespread, demands for delivery of such video data 
through a network have grown. 

Most of networks which have been widely available 
aslocsUWea netwo^ 

networks such as Ethernet These networks use 
CSM A/CD (Carrier Sense Multiple Access with Collision 
Detection) system mainly, and hence show the charac- 
teristics that increases the transmission delay time 
exponentially as the amount of data transferred 
increases. 

When video data are delivered in real time, it is nec- 
essary to maintain a constant transfer bit rate to ensure 
the continuity in delivery of video data. Even when using 
a common type network which causes a transmission 
delay dependent upon the amount of data, the transfer 
bit rate can be maintained by using a communications 
protocol designed specialty for video data to surety 
ensure the continuity in delivery of video data. 

However, when delivering video data using a com- 
mon type network, a mixture of the video data and other 
available data may cause cfifficutty in keeping the trans- 
fer bit rate of video data because of the traffic of the 
other available data even though a mechanism for main- 
taining the transfer bit rate of video data is added to the 
system. In such a case, the continuity in delivery of 
video data can not be ensured surely, and interrupts 
and disturbs are caused during the replay of video data 
transferred. 

Therefore, it is necessary to provide a cable 
intended for delivery of video data to prevent the video 
data from being mixed with other available data, in addi- 
tion to provision of a mechanism for maintaining the 
transfer bit rate of video data, to surely ensure the con- 
tinuity in delivery of video data when delivering the 
video data in real time using a common type network. 

A method of reducing the amount of video data to 
be delivered when the bandwidth of a network required 
for video data transmission cannot be ensured suffi- 
ciently is disclosed in Japanese Patent Application Laid 
Open (KOKAI) No. 7-75092. In accordance with this 
method, when compressing video data to be delivered 
through a network, the amount of data is reduced by 
changing spatial frequency region used according to a 



load imposed on the network. In addition, to ensure a 
constant bandwidth, a local area network using TDM 
(Time Division Multiplexer) is utilized as the network. 
Referring now to Fig. 9, it illustrates a block diagram 

5 showing a prior art system for delivering video data 
which is applied to a network with CSMA/CD system, as 
disclosed in Japanese Patent Application Laid Open 
(KOKAI) No. 7-75092. In Fig. 9, reference numeral 201 
denotes a terminal device, 204 denotes a network 20 

10 denotes a camera for inputting images, 22 denotes a 
pre-processing unit for, for example, digitizing an input 
signal, 23 denotes a discrete-cosine transformation 
unit, and 24 denotes a control unit for dynamically 
changing a spatial frequency. Furthermore, reference 

15 numeral 25 denotes a quantization unit, 26 denotes a 
variable-length coding unit, 21 denotes a microphone 
for generating an audio signal, 29 denotes a sound sig- 
nal processing unit, 27 denotes an image/sound multi- 
plexer, 28 denotes an interface unit between the 

20 terminal deviceahd the network 204, 32 denotes anet- 
work monitoring unit, 31 denotes a control unit for con- 
trolling the upper limit of transmission frequency, and 30 
denotes an unit for selecting transmission frequency 
components. 

25 Next, a description will be made as to the operation. 
The pre-processing unit 22 and discrete cosine trans- 
forming unit 23 digitize and convert an image input 
through the camera 20 into components in the spatial 
frequency region. In accordance with a load imposed on 

30 the network 204 detected by the network monitoring unit 
32, the transmission frequency upper limit controlling 
unit 31 and transmission frequency components select- 
ing unit 30 judge and select frequency components to 
be transferred. 

35 The quantization unit 25 quantizes the data about 
the range of frequency selected and the variable-length 
coding unit 26 encodes the quantized data into variable- 
length codes. The sound signal processing unit 29 digi- 
tizes an audio signal input through the microphone 21 

40 and compresses corresponding digital data. The 
image/sound multiplexer 27 combines the data from the 
variable-length coding unit 26 and sound processing 
unit 29 to generate a set of video data using time divi- 
sion multiplexing technique. The interface unit 28 con- 

45 verts the video data signal into the form suitable for th 
network 204. Finally, the video data is delivered to the 
network 204. 

Since the prior art video data delivering system 
having the structure mentioned above is based on the 

so premise that video data input in real time and com- 
pressed are transmitted through a common type net- 
work such as Ethernet, there is a problem in that, when 
delivering video data compressed and stored at a con- 
stant bit rate, it is necessary to reproduce video data in 

55 a state previous to the compression and compress the 
video data again according to a load imposed on the 
network. 
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SUMMARY OF THE INVENTION 

The present invention is made to overcome the 
aforementioned problem. More precisely, it is an object 
of the present invention to provide a system for, when 
delivering already compressed and stored video data 
through a network in which other available data can 
exist, adjusting the transfer bit rate of video data to be 
transmitted through the network in order to surely 
ensure the continuity in delivery of video data if the 
transfer bit rate cannot be maintained constantly 
because of a high load imposed on the network and 
hence the continuity in delivery of video data cannot be 
ensured. 

Furthermore, it is another object of the present 
invention to provide a system capable of eliminating the 
adverse effect of decreasing the transfer bit rate of other 
available data during delivery of video data. 

In accordance with one aspect of the present inven- 
tion, there is provided a system for delivering video data 
through a common type network including a video 
server which can deliver video data and a client which 
can receive and replay video data delivered thereto, the 
video server comprising: a video data index having 
video data index information required to set a transfer 
bit rate of video data to be delivered by the video server; 
a video data assembler for extracting data from an orig- 
inal video data by referring to the video data index infor- 
mation so as to set the transfer bit rate, and for 
reassembling the extracted data to create video data to 
be transferred at the set transfer bit rate to the client, 
while maintaining consistency in contents of the original 
video data; and a video data delivery unit for delivering 
a constant amount of data at certain intervals in accord- 
ance with the set transfer bit rate in order to transfer the 
video data reassembled by the video data assembler to 
the client, thereby making it possible for the client to 
replay the video data continuously. 

In a preferred embodiment of the present invention, 
the video index information defines a plurality of settings 
for the transfer bit rate of video data and indicates which 
data included in the original video data the video server 
should transfer to the client when setting the transfer bit 
rate to one of the plurality of settings. Furthermore, the 
video data assembler extracts data from the original 
video data by referring to the video data index informa- 
tion so as to set the transfer bit rate to one of the plural- 
ity of settings. 

The original video data can be compressed by 
using a predetermined compression method. Further- 
more, the video data index information can indicate at 
least types, numbers, and positions on a time axis of 
data which are to be selected from the compressed 
original video data by the video data assembling 
means. 

Preferably, the video server is adapted to analyze 
the original video data so as to create the video index 
information when the original video data are registered 
in the video server. 



In a preferred embodiment of the present invention, 
the video server is adapted to determine a load 
imposed on the network, and the video data assembler 
selects and extracts data to be transferred from the orig- 
5 inal video data in accordance with a measurement 
result of the network load. The video data assembler 
can compare the measurement result of the network 
load to a reference value of the network load associated 
with the transfer bit rate set to one of the plurality of set- 
to tings, and set the transfer bit rate to another of the plu- 
rality of settings on the basis of a comparison result as 
needed so as to increase or decrease the transfer bit 
rate. 

Preferably, the network load is determined by 

is measuring a time required for transfer of the reassem- 
bled video data to the client at certain intervals. Alterna- 
tively, a network load sensor can measure a load 
imposed on the network. 

Instead of determining a load imposed on the net- 

20 work, the video server can determine a load imposed 
thereon, and the video data assembler can select and 
extract data to be transferred from the original vide 
data in accordance with a measurement result of th 
video server load. The video data assembler compares 

25 the measurement result of the video server load to a ref- 
erence value of the video server load associated with 
the transfer bit rate set to one of the plurality of settings, 
and sets the transfer bit rate to another of the plurality of 
settings on the basis of a comparison result as needed 

30 so as to increase or decrease the transfer bit rate. 

In accordance with another aspect of the present 
invention, there is provided a system for delivering video 
data through a common type network including a video 
server which can deliver video data and a client which 

35 can receive and replay video data delivered thereto, 
wherein the video server comprises: a video data index 
having video data index information required to set a 
transfer bit rate of video data to be delivered by the 
video server; a video data assembler for extracting data 

40 from an original video data by referring to the video data 
index information so as to set the transfer bit rate, and 
for reassembling the extracted data to create video data 
to be transferred at the set transfer bit rate to the client, 
while maintaining consistency in contents of the original 

45 video data, and wherein the client comprises a pr - 
charge buffer for absorbing changes in the transfer bit 
rate of the video data delivered by the video server s 
as to prevent video data transferring services provided 
by the video server from being interrupted. 

so In a preferred embodiment of the present invention, 
the video index information defines a plurality of settings 
for the transfer bit rate of video data and indicates which 
data included in the original video data the video server 
should transfer to the client when setting the transfer bit 

55 rate to one of the plurality of settings. Furthermore, the 
video data assembler extracts data from the original 
video data by referring to the video data index informa- 
tion so as to set the transfer bit rate to one of the plural- 
ity of settings. 
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The original video data can be compressed by 
using a predetermined compression method. Further- 
more, the video data index information can indicate at 
least types, numbers, and positions on a time axis of 
data which are to be selected from the compressed 
original video data by the video data assembling 
means. 

Preferably, the video server is adapted to analyze 
the original video data so as to create the video index 
information when the original video data are registered 
in the video server. 

In a preferred embodiment of the present invention, 
the client is adapted to determine a toad imposed on the 
network, and the client informs the video server of a 
measurement result of the network load and the video 
data assembler selects and extracts data to be trans- 
ferred from the original video data in accordance with 
the measurement result of the network load. The video 
data assembler compares the measurement result of 
the network load to a" r^CTence~value of the network 
load associated with the transfer bit rate set to one of 
the plurality of settings, and sets the transfer bit rate to 
another of the plurality of settings on the basis of a com- 
parison result as needed so as to increase or decrease 
the transfer bit rate. 

Further objects and advantages of the present 
invention will be apparent from the following description 
of the preferred embodiments of the invention as illus- 
trated in the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing a system for deliv- 
ering video data according to the present invention; 
Rg. 2 is a block diagram of a video server according 
to a first embodiment of the present invention; 
Rg. 3 is a view of a table of a video index according 
to the present invention; 

Rg. 4 is a flow diagram showing the operation of the 
first embodiment of the present invention; 
Rg. 5 is a block diagram of a video server according 
to a second embodiment of the present invention; 
Rg. 6 is a flow diagram showing the operation of the 
second embodiment of the present invention; 
Rg. 7 is a block diagram of a video server according 
to a third embodiment of the present invention; 
Rg. 8 is a block diagram of a video data delivering 
system according to a fourth embodiment of the 
present invention; and 

Rg. 9 is a block diagram of a prior art video data 
delivering system. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

A description will be made as to a first embodiment 
of the present invention with reference to Figs. 1 
through 4. 

Rg. 1 shows a block diagram showing the structure 



of a system for delivering video data according to the 
present invention, Rg. 2 shows a block diagram of a 
video server shown in Rg. 1 , Fig. 3 shows a table show- 
ing an example of a video data index, and Rg. 4 shows 

5 a flow diagram showing the operation of a system of 
delivering video data according to the first embodiment. 
In Figs. 1 and 2. reference numeral 1 denotes the video 
server, 2 denotes a client which can receive video data 
from the video server 1 and replay the video, and 3 

ro denotes a common type network such as Ethernet. 

The video server 1 delivers video data to the client 
2 in response to a request of the client 2. The client 2 
receives and replays the video data delivered thereto by 
the video server 1 . In addition to the case where the cli- 

15 ent 2 receives video data from the video server 1 , ther 
may be cases where the client 2 receives or delivers 
data except video data from or to another client. 

In Rg. 2, reference numeral 11 denotes a video 
data database constructed by video data 12 and th 

20 video data index 13. The video data" 12 are digital video 
data which are compressed by using a compressi n 
method such as MPEG1 and are to be delivered by the 
video server 1. 

The video data index 13 is video data index infor- 

25 mation for describing parts which are selected from 
among the compressed video data 1 2 and are delivered 
actually, in order for the video server 1 to adjust the 
transfer bit rate at which the video data 12 are transmit- 
ted to the client 2 and deliver the video data without loss 

30 of the consistency in the contents of video data. 

Furthermore, reference numeral 14 denotes a 
video data assembler for extracting data in order to 
adjust the transfer bit rate from the video data 12 using 
the video data index 13 while maintaining the consist- 
as ency in the contents of video data, and for reassembling 
the extracted data to create video data to be transmitted 
at a transfer bit rate which can be different from the 
transfer bit rate of the original video data 12. 

Reference numeral 15 denotes a video data deliv- 

40 ery unit for delivering a constant amount of video data at 
constant intervals in accordance with the transfer bit 
rate of video data reassembled, and 16 denotes a bus 
through which data are exchanged between compo- 
nents within the video server 1 . 

45 The video data index 13 is created when the origi- 
nal video data 12 are registered in the video server 1. 
For example, when the video data are compressed in 
the form of MPEG1, each frame to be displayed is 
reconstructed from any one of three kinds of pictures, 

so i.e., an I picture, a P picture, and a B picture. In the case 
of I picture, each frame can be reconstructed from only 
an I picture. In the case of P picture, each frame can be 
reconstructed from a previous I picture or a previous P 
picture as well as a P picture. In the case of B picture, a 

55 previous I picture and a previous P picture are needed 
to reconstruct each frame as well as a B picture. 

A plurality of picture sets each constructed of three 
kinds of pictures construct a group called GOP. Fig. 3 
shows an example of the video data index when original 
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video data compressed in the form of MPEG1 are regis- 
tered in the video server. The video server 1 analyzes 
the original video data 12 to create the video data index 
13 when registering the video data 12 in the video 
server. As shown in Fig. 3, the video data index defines 
a plurality of settings for the transfer bit rate of video 
data, an original bit rate, a 1 st bit rate, a 2nd bit rate, 
and a nth bit rate, and indicates which data included in 
the original video data 12 the video server 1 should 
transfer to the client 2 when setting the transfer bit rate 
to one of the plurality of settings. More specifically, the 
video data index 1 3 indicates the types, numbers, and 
positions on a time axis of data which are to be selected 
from the compressed original video data by the video 
data assembler 14. In the example shown in Fig. 3, 
types of pictures to be delivered and positions on a time 
axis of the pictures in the video data 12 are described 
for each of the plurality of bit rate settings. For example, 
all the pictures are delivered in the case of the original 
bit rate setting, and all the I and P pictures are delivered 
in the case of the first bit rate setting. 

Next, a description will be made as to the video 
data transferring operation with reference to Figs. 2 and 
4. 

In Rg. 2, the client 2 outputs a request for transfer 
of video data to the video server 1 . When the video 
server 1 receives the transfer request from the client, 
the video server delivers a constant amount of data 
within a certain time in accordance with the transfer bit 
rate of video data so as to transfer the video data 
requested by the client through the video data delivering 
unit 15, in steps F41 and F42. 

For example, when video data are transferred at a 
transfer bit rate of 1 .5 Mbps, 64 Kbyte of data are deliv- 
ered every 340 msec. If there is a margin of the load 
imposed on the network, the system can maintain the 
transfer bit rate of 1 .5 Mbps for these delivery parame- 
ters. However, the video server cannot maintain the 
transfer bit rate if the load is increased. In order to 
resolve this problem, the video server starts to measure 
a time T r required for transfer of data at constant inter- 
vals T<\ so as to determine the load imposed on the net- 
work just after transfers of video data are started. Then, 
the video server, in steps F43 and 45, compares a 
measured value T r to a reference value T d1 , which is the 
maximum of a time required for transfer of data that can- 
not be exceeded in order to maintain the current transfer 
bit rate. 

If the measured value T r exceeds the reference 
value T d1 , the video data assembler 14, in step F44, 
extracts all the I pictures and P pictures defined as 
video data to be transferred at the second transfer bit 
rate from the video data 12 so as to set the transfer bit 
rate to the second bit rate setting, modifies the header 
information in such a manner that the information shows 
that video data to be transferred are constructed of all 
the I and P pictures, and reassembles the extracted 
data to create video data to be transferred at the second 
transfer bit rate which is reduced from the original trans- 



fer bit rate by one level. The video data delivery unit 15 
delivers the video data created at the new transfer bit 
rate while creating the video data rather than delivers all 
video data after they are created from the video data 1 2 . 

5 During transfers of video data, the video server 
continues to measure a time T r required for transfer of 
data at constant intervals T v Every when a measured 
value T r exceeds a reference value T d associated with 
the current transfer bit rate, the video data assembler 14 

10 repeats the similar operation. The reference value is 
predetermined for each of the plurality of transfer bit rate 
settings described in the video data index 13. When the 
video data assembler 14 judges that it is necessary to 
transfer video data by reducing the transfer bit rate by 

is one level, however, the current transfer bit rate is set to 
the lower limit of the settings described in the video data 
index, it continues to transfer the video data at the cur- 
rent transfer bit rate which remains unchanged, in step 
F46. 

20 On the other hand, if measured values of the data 
transferring time T r are smaller than a reference value 
T d during a time T s after the transfer bit rate is changed, 
in step F47, the video data assembler 14, in step F49, 
extracts data to be transferred at a transfer bit rat . 

25 which is increased from the current transfer bit rate by 
one level in accordance with the video data index so as 
to increase the current bit rate and reassembles the 
extracted data to create video data. Then, the video 
data delivery unit 15 delivers the video data. For exam- 

30 pie, when the current transfer bit rate is increased by 
one level from the second bit rate to the first bit rate as 
described in the video data index, the video data 
assembler 14 extracts all the I pictures and P pictures 
from the video data 12, modifies the header information, 

35 and reassembles the extracted data to create video 
data to be transferred through the video data delivery 
unit 15. In this case, since the new transfer bit rate for 
the video data created is 1 Mbps, as shown in Fig. 3. the 
video data delivery unit delivers 64 kbyte of data every 

40 512 msec. If the video data assembler 14, in step F48, 
judges that it is necessary to increase the transfer bit 
rate by one level, however, the current bit rate has been 
already set to the original bit rate setting, the video 
server continues to transfer the video data at the current 

45 bit rate which remains unchanged. 

The video server repeats the above-mentioned 
successive operations such as measurements of th 
data transferring time, and increasing and decreasing 
the transfer bit rate until it delivers all video data needed 

so in order for the client to replay the original video data 12 
while maintaining the consistency in the contents of the 
video data. 

A description will be made as to a second embodi- 
ment of the present invention with reference to Figs. 5 
55 and 6. 

Rg. 5 shows a block diagram of the video server 1 
according to the second embodiment In Fig. 5, refer- 
ence numeral 17 denotes a network load sensor for 
detecting a load imposed on the network. The other 
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components are the same as those designated by the 
same reference numerals shown in Fig. 2. 

Next, a description will be made as to the video 
data transferring operation with reference to Figs. 3, 5 
and 6. 

In Ftg. 5, the client 2 outputs a request for transfer 
of video data to the video server 1. When the video 
server receives the transfer request from the client, the 
video server delivers a constant amount of data within a 
certain time in accordance with the transfer bit rate of 
video data so as to transfer the video data requested by 
the client through the video data delivering unit 15. in 
steps F61 and F62. 

The video server, in step F63, starts to measure a 
load Lp imposed on the network at constant intervals T 1 
by means of the network load sensor 1 7 just after trans- 
fers of video data are started. Then, the video server 
compares a measured value Lp of the network load to a 
reference value L^, which is the maximum of the net- 
work load that carmof be^ exceeded iForder tolhaintain 
the current transfer bit rate. 

If the measured value U exceeds the reference 
value L^, the video data assembler 14, in step F64, 
selects a bit rate setting which can cause a measured 
value Lp to become lower than the reference value L d1 
from the video data index 1 3. The video data assembler 
14 further extracts pictures from the video data 12 in 
accordance with the bit rate setting selected from the 
video data index, modifies the header information, and 
reassembles the extracted data to create video data to 
be transferred at the selected bit rate which is smaller 
than the current transfer bit rate. Then, the video data 
delivery unit 15 delivers the video data created at the 
new transfer bit rate. The reassembling of video data to 
be transferred at the reduced transfer bit rate and trans- 
fer of video data are carried out in parallel. The video 
data delivery unit 1 5 delivers a constant amount of data 
at certain intervals so as to maintain the current transfer 
bit rate when transferring video data. After that, the 
video data assembler 14, in step F65, continues to 
determine whether or not a measured value Lp of the 
network load exceeds a reference value L^, which is 
associated with the current transfer bit rate set to one of 
the plurality of bit rate settings. And, every when a 
measured value L„ of the network load exceeds the cor- 
responding reference value L^, the video data assem- 
bler 14 changes the bit rate in the same manner. The 
reference value L$ is defined for each of the plurality of 
transfer bit rate settings. 

When the measured value Lp exceeds the refer- 
ence value Ld, the video data assembler 14 checks 
whether or not the current transfer bit rate is set to the 
lower limit of the plurality of transfer bit rate settings 
described in the video data index, ff the current transfer 
bit rate is the lower limit, the video data assembler 1 4, in 
step F68, continues to transfer video data at the current 
transfer bit rate which remains unchanged. 

On the other hand, if measured values Ln are 
smaller than the reference value during a predeter- 



mined time T s after the transfer bit rate was changed in 
step F67, the video data assembler 14, in step F69, 
selects the maximum from among transfer bit rate set- 
tings defining reference values Id which are equal to or 

5 larger than the measured values Lp obtained after the 
previous change of the bit rate was completed and reas- 
sembles data extracted to create video data to be trans- 
ferred at the selected bit rate in the aforementioned 
manner. Then, the video data delivery unit 15 delivers 

yo the video data. Before performing step F69, the video 
data assembler, in step F68, determines whether or not 
the current bit rate is set to the original bit rate setting. If 
the current bit rate is set to the original bit rate setting, 
the video server continues to transfer the video data at 

is the current bit rate which remains unchanged. 

The video server repeats the above-mentioned 
successive operations such as measurements of the 
data transferring time, and increasing and decreasing 
the transfer bit rate until it delivers all video data needed 

20 Hn aiieTfoFthircii^ 

while maintaining the consistency in the contents of the 
video data. 

A description will be made as to a third embodiment 
of the present invention with reference to Fig. 7. 

25 Fig. 7 shows a block diagram of the video server 
according to this embodiment. In the figure, reference 
numeral 1 8 denotes a server bad sensor for detecting a 
load imposed on the video server. The other compo- 
nents are the same as those designated by the same 

30 reference numerals shown in Fig. 2. 

Next, a description will be made as to the operation. 
The video server according to this embodiment meas- 
ures a load imposed on the video server by means of 
the server load sensor 18 instead of measuring a time T r 

35 required for transfer of data at constant intervals T1 , as 
described in the first embodiment. 

Furthermore, in this embodiment, the video server 
is adapted to perform similar operations like the first 
embodiment, by replacing reference values T d of a time 

40 required for transfer of data as mentioned in the first 
embodiment with reference values 1^ of a load imposed 
on the video server, which are respectively associated 
with a plurality of transfer bit rate settings defined in the 
video data index 13. 

45 A description will be made as to a fourth embodi- 
ment of the present invention with reference to Fig. 8. 
Fig. 8 shows a block diagram of a system for delivering 
video data according to the fourth embodiment. In Fig. 
8, reference numeral 111 denotes a video databas 

so constructed by video data 112 to be transferred by a 
video server 101 and a video data index 113 including 
index information for indicating parts which are selected 
from the video data 112 and are delivered actually. 
Furthermore, reference numeral 114 denotes a 

55 video data assembler adapted to extract data required 
to adjust the transfer bit rate from the original video data 
1 12 while maintaining the consistency of the contents of 
the original video data, and reassemble the extracted 
data to create video data to be transferred. For example, 
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in the case that the video data 1 12 are compressed in 
the form of MPEG1 , information about the frame struc- 
ture such as the type of encoding, the size of the 
header, the size of frames, and the number of pictures 
which construct GOP, the types and numbers of pictures 
to be extracted which are predetermined for each of a 
plurality of transfer bit rate settings, and the positions on 
a time axis of the pictures within the video data 12 are 
described in the video data index 13 when the video 
data are registered in the video server. Furthermore, 
picture degradation in the video data followed by a 
change in the transfer bit rate is prevented by determin- 
ing the types and numbers of pictures to be extracted for 
each of the plural bit rate settings by assigning a weight 
to each of pictures which construct a GOP in accord- 
ance with the contents of the video data every when 
registering the video data. That is, the structure of the 
video data assembler 114 is the same as that of the 
video data assembler 14 according to the aforemen- 
tioned embodiments, and the contents of the video data 
index 1 13 are similar to those of the video data index 13 
according to the aforementioned embodiments. 

Reference numeral 117 denotes a network load 
sensor for detecting a load imposed on a network 103, 
and 121 denotes a precharge buffer, which is a compo- 
nent of the client 102. for absorbing changes in the 
transfer bit rate of video data delivered by the video 
server 1 01 so as to prevent video data transferring serv- 
ices provided by the video server 101 from being inter- 
rupted. 

Next, a description will be made as to the video 
data transferring operation in the fourth embodiment. 
First, the client 102 outputs a request for transfer of 
video data to the video server 101. When the video 
server 101 receives the transfer request from the client, 
the video server transfers the video data requested by 
the client 102 to the client. The client 102 stores the 
video data delivered by the video server in the pre- 
charge buffer 121. and then starts to replay the video 
data. After that, the client continues to store video data 
received in the precharge buffer and read the video data 
to be replayed from the precharge buffer. Thus, even 
though transfer of video data from the video server is 
delayed, the client can maintain the continuity in video 
replay by absorbing changes in the transfer bit rate of 
video data by utilizing video data stored in the pre- 
charge buffer. 

The network load sensor 1 17 starts to measure a 
load Ln imposed on the network at constant intervals 
just after transfers of video data are started. Then, the 
network load sensor 1 17 compares the measured net- 
work load l_n to a reference value 1^. If the measured 
value Lp exceeds the reference vatue L^, the client 102 
sends a request to reduce the transfer bit rate to the 
video server 101. When the video server 101 receives 
the request to change the bit rate, the video data 
assembler 1 1 4 extracts pictures from the video data 1 1 2 
In accordance with the video data index 113, modifies 
the header information, and reassembles the extracted 



data to create video data to be transferred at a bit rate 
which is reduced from the previous bit rate by one level 
from the video data 12. Then, the video server deliv rs 
the video data. 

5 If it is necessary to reduce the bit rate by one level, 
however, the current transfer bit rate is set to the low r 
limit of the plural transfer bit rate settings defined in the 
video data index 113, the video server continues to 
transfer video data at the current transfer bit rate which 

io remains unchanged. 

After that, every when a measured value Ln 
exceeds the reference value associated with the cur- 
rent transfer bit rate, the client sends a request to 
reduce the bit rate to this video server and the video 

is server repeats simitar operations to reassemble data 
extracted to create video data to be transferred at a bit 
rate which is reduced from the previous bit rate by one 
level and then deliver the video data. 

On the other hand, if measured values are 

20 smaller than the reference value Lc during a tim T s 
after the transfer bit rate was changed, the dient sends 
a request to increase the transfer bit rate to the video 
server. When the video server receives the request to 
change the bit rate, the video data assembler 114 

25 extracts pictures from the video data 1 1 2 in accordance 
with the video data index 13. modifies the header infor- 
mation, and reassembles the extracted data to create 
video data to be transferred at a bit rate which is 
increased from the previous bit rate by one level. Then, 

30 the video server delivers the video data. If it is neces- 
sary to increase the transfer bit rate by one level, how- 
ever, the current bit rate is set to the original bit rate 
setting, the video server continues to transfer the video 
data at the current bit rate which remains unchanged. 

35 The client and video server repeat the abovemen- 
tioned successive operations such as detection of a 
load imposed on the network, and increasing and 
decreasing the transfer bit rate until it delivers all video 
data needed in order for the client to replay the original 

40 video data 1 12 while maintaining the consistency in the 
contents of the video data. 

As previously mentioned, in accordance with the 
present invention, since there is provided a video serv r 
which dynamically changes the transfer bit rate in 

45 response to a load imposed on a resource such as a 
network or the video server when delivering video data, 
the present invention offers an advantage in its capabil- 
ity to maintain the continuity in video data to be deliv- 
ered even though the video server delivers video data 

50 through a common type network in which the video data 
and other available data may be mixed and a certain 
bandwidth cannot be ensured in advance. 

Furthermore, the present invention provides an 
advantage in eliminating an adverse effect on transfers 

55 of other available data when delivering video data which 
may exist together with the other available data in the 
network. 

Many widely different embodiments of the present 
invention may be constructed without departing from 



7 



BNSDOCID: <EP 0798924A2J_> 



13 



EP 0 798 924 A2 



14 



the spirit and scope of the present invention. It should 
be understood that the present invention is not limited to 
the specific embodiments described in the specification, 
except as defined in the appended claims. 

5 

Claims 

1. A system for delivering video data through a com- 
mon type network (3) including a video server (1) 
which can deliver video data and a client (2) which w 
can receive and replay video data delivered thereto, 
characterized in that the video server comprises: 

a means (11) for storing video index informa- 
tion (13) required to set a transfer bit rate of is 
video data to be delivered by the video server; 
a video data assembling means (14) for 
extracting data from an original video data (12) 
by referring to the video data index information 
so as~to sef the transfer bit rat^arsd for reas-" 26 
sembling the extracted data to create video 
data to be transferred at the set transfer bit rate 
to the client, while maintaining consistency in 
contents of the original video data; and 
a video data delivery means (15) for delivering 25 
a constant amount of data at certain intervals in 
accordance with the set transfer bit rate in 
order to transfer the video data reassembled by 
the video data assembling means to the client, 
thereby making it possible for the client to 30 
replay the video data continuously. 

2. The system according to Claim 1 , characterized in 
that the video index information defines a plurality 

of settings for the transfer bit rate of video data and 35 
indicates which data included in the original video 
data the video server should transfer to the client 
when setting the transfer bit rate to one of the plu- 
rality of settings, and wherein the video data 
assembling means extracts data from the original 40 
video data by referring to the video data index infor- 
mation so as to set the transfer bit rate to one of the 
plurality of settings. 

3. The system according to Claim 2, characterized in 45 
that the original video data are compressed by 
using a predetermined compression method, and 
wherein the video data index information indicates 

at least types, numbers, and positions on a time 
axis of data which are to be selected from the com- so 
pressed original video data by the video data 
assembling means. 

4. The system according to any one of Claims 1 to 3. 
characterized in that the video server further com- 55 
prises a video data analyzing means for analyzing 
the original video data so as to create the video 
index information when the original video data are 
registered in the video server. 



5. The system according to Claim 2 or 3, character- 
ized in that the video server comprises a network 
load measuring means for determining a load 
imposed on the network, and wherein the video 
data assembling means selects and extracts data 
to be transferred from the original video data in 
accordance with a measurement result of the net- 
work load by the network load determining means. 

6. The system according to Claim 5, characterized in 
that the video data assembling means compares 
the measurement result of the network load to a ref- 
erence value of the network load associated with 
the transfer bit rate set to one of the plurality of set- 
tings, and sets the transfer bit rate to another of th 
plurality of settings on the basis of a comparison 
result as needed so as to increase or decrease the 
transfer bit rate. 

7. The^ystem accordingto" Claim 5 or 6rcharacter- 
ized in that the network load measuring means is a 
means for measuring a time required for transfer of 
the reassembled video data to the client at certain 
intervals. 

8. The system according to Claim 5 or 6, character- 
ized in that the network load measuring means is a 
network load sensor (17) for measuring a load 
imposed on the network. 

9. The system according to Claim 2 or 3, character- 
ized in that the video server comprises a server 
toad measuring means (18) for determining a load 
imposed on the video server, and wherein the video 
data assembling means selects and extracts data 
to be transferred from the original video data in 
accordance with a measurement result of the video 
server load by the server load measuring means. 

10. The system according to Claim 9, characterized in 
that the video data assembling means compares 
the measurement result of the video server load to 
a reference value of the video server load associ- 
ated with the transfer bit rate set to one of the plu- 
rality of settings, and sets the transfer bit rate to 
another of the plurality of settings on the basis of a 
comparison result as needed so as to increase r 
decrease the transfer bit rate. 

11. A system for delivering video data through a com- 
mon type network (103) including a video server 

(101) which can deliver video data and a client 

(102) which can receive and replay video data 
delivered thereto, characterized in that the video 
server comprises: 

a means (11 1) for storing video index informa- 
tion (113) required to set a transfer bit rate of 
video data to be delivered by the video server; 
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and 

a video data assembling means (114) for 
extracting data from an original video data 
(1 12) by referring to the video data index infor- 
mation so as to set the transfer bit rate, and for s 
reassembling the extracted data to create 
video data to be transferred at the set transfer 
bit rate to the client, while maintaining consist- 
ency in contents of the original video data, and 
characterized in that the client comprises: 10 
a precharge buffer (121) for absorbing changes 
in the transfer bit rate of the video data deliv- 
ered by the video server so as to prevent video 
data transferring services provided by the 
video server from being interrupted. is 

12. The system according to Claim 1 1 , characterized in 
that the video index information defines a plurality 
of settings for the transfer bit rate of video data and 
indicates which data included in the original video 20 
data the video server should transfer to the client 
when setting the transfer bit rate to one of the plu- 
rality of settings, and the video data assembling 
means extracts data from the original video data by 
referring to the video data index information so as to 25 
set the transfer bit rate to one of the plurality of set- 
tings. 

13. The system according to Claim 12 t characterized in 
that the original video data are compressed by 30 
using a predetermined compression method, and 
the video data index information indicates at least 
types, numbers, and positions on a time axis of 
data which are to be selected from the compressed 
original video data by the video data assembling 35 
means. 



tings, and sets the transfer bit rate to another of the 
plurality of settings on the basis of a comparison 
result as needed so as to increase or decrease the 
transfer bit rate. 



14. The system according to any one of Claims 11 to 
13, characterized in that the video server further 
comprises a video data analyzing means for ana- 
lyzing the original video data so as to create the 
video index information when the original video 
data are registered in the video server. 



40 



15. The system according to Claim 12 or 13, character- 
ized in that the client comprises a network load 
measuring means (117) for determining a load 
imposed on the network, and the client informs the 
video server of a measurement result of the net- 
work load and the video data assembling means 
selects and extracts data to be transferred from the 
original video data in accordance with the measure- 
ment result of the network load. 



so 



1 6. The system according to Claim 1 5. characterized in 
that the video data assembling means compares 
th measurement result of the network load to a ref- 
erence value of the network load associated with 
the transfer bit rate set to one of the plurality of set- 



55 



9 



BNSDOCID: <EP 0798924A2J_> 



EP 0 798 924 A2 



FIG. 1 



VIDEO SERVER 



CLIENT 



CLIENT 



FIG. 2 



14 



VIDEO SERVER 



1 



VIDEO DATA 
ASSEMBLER 



12 



VIDEO 
DATA 



13 
I 



L 



i i 



VIDEO DATA 
INDEX 



VIDEO DATA 
DELIVERY UNIT 



1 6 



— 15 



7 



CLIENT 



BNSOOCID. <EP 0798924A2_I_> 



10 



EP 0 798 924 A2 



FIG. 3 



TYPE OF ENCODING 


MPEG1 


HEADER INFORMATION 


SIZE 


FRAME INFORMATION 


THE NUMBER OF PICTURED 
WHICH CONSTRACT ONE GOP 


ORIGINAL BIT RATE 
SETTING (1. 5Mbps) 


AT T O T PTTTDPC 


FIRST BIT RATE 
SETTING (1. OMbps) 


ALL I AND P PICTURES 


SECOND BIT RATE 
SETTING (768Kbps) 


ALL I P ICTURES. AND 
THREE P PICTURES 
FOR EVERY ONE GOP 


• • • • 




Nth BIT RATE 
SETTING (128Kbps) 


ONE I PICTURE FOR 
EVERY FIVE GOPS 



FIG. 5 



14 



VIDEO SERVER 



VIDEO DATA 
ASSEMBLER 



r!2 



VIDEO 
DATA 



13 



r 



1 1 



VIDEO DATA 
INDEX 



VIDEO DATA 
DELIVERY UNIT 

T~ 

15 



NETWORK 
LOAD SENSOR 



16 



17 



CLIENT 



11 



EP 0 798 924 A2 



FIG. 4 




V 



BNSDOCID: <EP 0798924A2_I_> 



12 



EP0 798 924 A2 



F61 



F62 



START TRANSFER 
OF VIDEO DATA 



FIG. 6 



TRANSFER VIDEO DATA 
AT THE ORIGINAL BIT 
RATE SETTING 



F63 



A MEASURED 
TALUE BY THE NETWORK 
LOAD SENSOR EXCEEDS A 
REFERENCE VALUE, 
«? 



NO 



YES 



F64 



1 



TRANSFER VIDEO 
DATA BY CHANGING 
THE BIT RATE SO 
THAT A MEASURED 
VALUE OF THE 
NETWORK LOAD IS 
LOWER THAN THE 
REFERENCE VALUE 



F65 



A 

MEASURED 
TALUE BY THE' 
NETWORK LOAD 
SENSOR EXCEEDS 
.A REFERENCE 
VALUE 
<> 



NO 



F67 



r A CERTAIN" 
TIME HAS 
ELAPSED SINCE 
A CHANGE 
OF THE BIT 
RATE ? 



NO 



YES 



F68 



THE BIT 
RATE IS SET TO^ 
'THE ORIGINAL BIT 
RATE SETTING 
«? 



YES 



F66 



YES 



NO 



THE BIT 
'RATE IS SET^ 
.TO THE LOWER 
LIMIT ? 



YES 



NO 



F69 



CHANGE THE BIT RATE TO 
THE MAXIMUM AMONG BIT 
RATE SETTINGS EACH OF 
WHICH DEFINES A 
REFERENCE VALUE 
WHICH IS EQUAL TO OR 
LARGER THAN MEASURED 
VALUES OF THE NETWORK 
LOAD . AND TRANSFER 
VIDEO DATA 



13 



0798924A2 I > 



EP 0 798 924 A2 



FIG. 7 



14 



VIDEO SERVER 



rl2 



2 



VIDEO DATA 
ASSEMBLER 



VIDEO 
DATA 



13 



r 



1 1 



VIDEO DATA 
INDEX 



VIDEO DATA 
DELIVERY UNIT 



SERVER 
LOAD SENSOR 



1 5 



16 



1 8 



CLIENT 



101 



FIG. 8 



1 14 



VIDEO SERVER 



VIDEO DATA 
ASSEMBLER 



112 



VIDEO 
DATA 



113 



111 



VIDEO DATA 
INDEX 



3 



^ io; 

















NETWORK 
LOAD SENSOR 




PRECHARGE 
BUFFER 




CLIENT 


^121 



102 



BNSDOCID. <EP 079892 4A2_I_> 



14 



EP0 798 924 A2 




15 

BNSDOCID: <EP 0796924A2_I_> 



THIS PAGE BUNK (uspto) 



(19) 



J 



Europaisches Patentamt 
European Patent Office 
Off i e uropeen d s brevets 




(11) 



EP 0 798 924 A3 



(12) 



EUROPEAN PATENT APPLICATION 



(88) 


U316 ot puDiicauon MO. 


«1\ intri? H04N 7/173 H04N 5/00 




05.09.2001 Bulletin 2001/36 




(43) 


riato rtf ni iKlir*atinn AP' 
UdLc Ul pULIIlLrdllUl l n£.. 






m 10 1QQ7 Bulletin 1997/40 

Ul. IU. 199/ DUIltSlin lasf'IW 




(21) 


Application number: 96115236.0 




(22) 


Date of filing: 23.09.1996 




(84) 


Designated Contracting States: 


(72) Inventors: 


DE FR GB 


• Ito, Teruaki 






2-chome, Chiyoda-ku, Tokyo 100 (JP) 


(30) 


Priority: 29.03.1996 JP 7749496 


• Yasoda, Gyota 


2-chome, Chiyoda-ku, Tokyo 100 (JP) 


(71) 


Applicant: MITSUBISHI DENKI KABUSHIKI 




KAISHA 


(74) Representative: Pfenning, Meinig & Partner 




Tokyo 100 (JP) 


Mozartstrasse 17 




80336 Munchen (DE) 



(54) System for delivering video data 

(57) A system for delivering video data comprises a 
video server (1 ) including a video database (1 1 ) for stor- 
ing video index information (13) defining a plurality of 
settings for a transfer bit rate of video data and indicating 
which data included in original video data (1 2) the video 
server should transfer to a client (2) through a network 
(3) when setting the transfer bit rate to one of the plu- 
rality of settings, a video data assembler (1 4) for extract- 
ing data from the original video data by referring to the 
video data index information so as to set the transfer bit 



rate to one of the plurality of settings, and for reassem- 
bling the extracted data to create video data to be trans- 
ferred at the set transfer bit rate to the client, while main- 
taining consistency in contents of the original video data, 
and a video data delivery unit (15) for delivering a con- 
stant amount of data at certain intervals in accordance 
with the set transfer bit rate in orderto transfer the video 
data reassembled by the video data assembler to the 
client. Furthermore, the video server comprises a net- 
work load sensor (17) for measuring a load imposed on 
the network. 



FIG. 2 



14 



VIDEO SERVER 



12 



13 



-1 1 



VIDEO DATA 
ASSEMBLER 



1 — 




1 VIDEO 
1 DATA 




VIDEO DATA 
INDEX 









CO 

< 

CM 
O) 

GO 

o> 

O 

Q_ 
IU 



VIDEO DATA 
DELIVERY UNIT ~15 



T 



16 



CLIENT 



Printed by Jouve, 7S001 PARIS (FR) 



BNSDOCID: <EP 0798924A3_I_> 



EP 0 798 924 A3 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 96 11 5236 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document with indication, where appropriate. 

of r elevant passages 



CHANG E ET AL: "VARIABLE BIT RATE MPEG 
VIDEO STORAGE ON PARALLEL DISK ARRAYS" 
PROCEEDINGS OF THE INTERNATIONAL WORKSHOP 
ON COMMUNITY NETWORKI NGI NTEGRATED 
MULTIMEDIA SERVICES TO THE H0ME,XX t XX, 

13 July 1994 (1994-07-13), pages 127-137, 
XP000924893 

* page 128, left-hand column, line 37 - 
right-hand column, line 13 * 

* subsection 7.1 -Frame -replacement -and- - 
retrieval" of section 7 "Scalability" * 

* table 6 * 

* abstract * 

ANONYMOUS: "Method to Deliver Scalable 
Video across a Distributed Computer 
System" 

IBM TECHNICAL DISCLOSURE BULLETIN, US, IBM 
CORP. NEW YORK, 

vol. 5, no. 37, 1 May 1994 (1994-05-01), 
pages 251-256, XP002079392 
ISSN: Q018-8689 

* the whole document * 

ELEFTHERIADIS A ET AL: "CONSTRAINED AND 
GENERAL DYNAMIC RATE SHAPING OF COMPRESSED 
DIGITAL VIDEO" 

| PROCEEDINGS OF THE INTERNATIONAL 

; CONFERENCE ON IMAGE PROCESSING. 

I (ICIP) , US, LOS ALAMITOS, IEEE COMP. SOC. 

| PRESS, 

23 October 1995 (1995-10-23), pages 
396-399, XP000603910 
ISBN: 0-7803-3122-2 

* section 1 "Introduction 0 * 

* section 2 "Dynamic rate shaping" * 

* abstract * 



-/-- 



The present search report has been drawn up for all claims 



Relevant 
to claim 



1-16 



CLASSIFICATION OF THE 
APPLICATION (UH.CI.6) 



H04N7/173 
H04N5/00 



1-16 



1-16 



TECHNICAL FIELDS 
SEARCHED (lntQ.6) 



H04N 



Ptaon ol search 

THE HAGUE 



Dale of ccmote&on o< tne searcn 

16 July 2001 j La, V 



CATEGORY OFCfTED DOCUMENTS 

X : particularly reJevara if takon alone 

Y : particularly relevant if combined with another 

document of trie same catogory 
A : technaJogrcal background 
O : non-written disclosure 
P : intermediate document 



T : th«ory or principle underlying the invention 
E : earlier patent document but published on, or 

after the fifing date 
D : document cited in the application 
L document cited lor other reasons 

& : member of the same patent family, corresponding 
document 



BNSDOCID: <EP 0798924A3J_> 



EP 0 798 924 A3 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 96 11 5236 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 

A 



Citation of document with indication, where appropriate, 
of relevant passages 



US 5 115 429 A (HLUCHYJ MICHAEL G 
19 May 1992 (1992-95-19) 
* abstract * 



ET AL) 



CHEN M -S ET AL: "USING RATE STAGGERING 
TO STORE SCALABLE VIDEO DATA IN A 
DISK-ARRAY-BASED VIDEO SERVER" 
PROCEEDINGS OF THE SPIE, 
6 February 1995 (1995-G2-06) , XP000575905 

* the whole document * 

CHEN M -S ET AL: "STORAGE AND RETRIEVAL 
METHODS TO SUPPORT FULLY INTERACTIVE 
P LAYOUT IN A DISK-ARRAY-BASED VIDEO 
SERVER* 

MULTIMEDIA SYSTEMS, DE, SPRINGER VERLAG, 
vol. 3, no. 3, 1 July 1995 (1995-07-01), 
pages 126-135, XPOGG561391 
ISSN: 0942-4962 

* page 126, right-hand column, line 12 - 
line 29 * 



Relevant 
to claim 



1-16 



1-16 



11 



The present search report has been arawn up lor all claims 



Place of ooarcM 

THE HAGUE 



Oslo ol completion of Iho soarch 

16 July 2001 



CLASSIFICATION OF THE 
APPLICATION (tnt.Q.6) 



"1 



TECHNICAL RELOS 
SEARCHED <tnLC1.6) 



La,' V 



CATEGORY OF CITED DOCUMENTS 

X : particularly relevant If taken atone 

Y : particularly relevant tf combined with another 

document ol the same category 
A : technological background 
0 : non-written disclosure 
P : intermediate document 



T : theory or principle underlying the invention 
E : earlier patent document, bul published on . or 

after the filing date 
D : document cited in the application 
L : document cited for other reasons 

& : member of the same patent family, corresponding 
document 



BNSDOCID: <EP 0798924A3_L> 



3 



EP 0 798 924 A3 



ANNEX TO THE EUROPEAN SEARCH REPORT 

ON EUROPEAN PATENT APPLICATION NO. EP 96 11 5236 



This annex lists the patent family members relating to the patent documents cited in the above-mentioned European search report. 
The members are as contained m the European Patent Office EDP file on 

The European Patent Office is in no way liable for these particulars which are merely given for the purpose ol information. 

16-07-2001 



Patent dccumem 
cited in search report 


Publication 
date 


Patent family 
memberts) 


Publication 
date 


US 5115429 A 


19-05-1992 


AU 


636056 B 


08-04-1993 






AU 


8287191 A 


02-03-1992 






CA 


2066406 A,C 


03-02-1992 






DE 


69130559 D 


14-01-1999 






DE 


69130559 T 


15-07-1999 






EP 


0495036 A 


22-07-1992 






JP 


5502777 T 


13-05-1993 






WO 


9202998A 


2D-02-1992" 



i 



I 

a 

a 
O 

u. 1 , _ . _ ________________ 

O 

£. For more details about this annex : see Official Journal ol the European Patent Office, No. 1 2/82 



BNSOOCID: <EP 0798924A3_I_> 



4 



